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Abstract 
Problem/Purpose: Young children with developmental speech and/or language impairment 
(DSLI) often fail to develop important oral language and early literacy skills that are 
foundational for subsequent schooling and reading success. The purpose of this investigation was 
to examine the efficacy of the TELL curriculum and associated evidence-based teaching 
practices in promoting the acquisition of oral language and early literacy skills for preschool 
children with DSLI. 
Participants: Participants included 202 male and 87 female preschoolers with DSLI in the 
absence of other developmental impairment. Children ranged in age from 43 to 63 months. They 
were enrolled in 91 inclusive preschool classes and their corresponding classroom teachers were 
all female. 
Method: In this cluster RCT, classroom teachers were randomly assigned to implement the 
TELL curriculum or to continue with their business-as-usual (BAU) curriculum. Proximal 
outcomes were assessed with investigator-developed curriculum-based measures (CBM) 
administered six times over the school year and an investigator-developed assessment of 
vocabulary targeted in TELL. Standardized tests of oral language (Clinical Evaluation of 
Language Fundamentals Preschool - 2™ Edition), and early literacy skills (Test of Preschool 
Early Literacy), and a benchmarked early literacy assessment (Phonological Awareness and 
Literacy Screening PreK) were administered at the beginning and end of the school year to 
determine impact on more distal outcomes. 
Results: Results indicated a significant TELL effect for all CBMs at later measurement points 


with Cohen’s ds in the medium (.43) to very large (1.25) range. TELL effects were also noted for 


the vocabulary measures with small to medium between-group effect sizes (Cohen’s f*2 range 
from .02 to .44). There were no significant TELL effects for the more distal measures. 
Conclusion: Based on progress measures, the TELL curriculum was effective for improving the 


oral language and early literacy skills of young children with DSLI. 


Research Highlights 


1. 


Teaching Early Literacy and Language (TELL) is a tier one curriculum that was 
developed for young children with developmental disabilities and tested in an RCT with 
91 teachers and the 289 children with developmental speech and/or language impairment 
who were enrolled in their preschool classes. 

Curriculum-based measures developed for TELL were administered to all children at six 
time points across the school year. 

Growth curve estimates indicated increased magnitude of growth for children who 
received TELL with peaks outside the scope of the model for all oral language and 
eagerly literacy skills that were measured. Children assigned to the BAU classes 
typically demonstrated a plateau. 

Comparison of performance at each CBM time point for each skill indicated that children 
in TELL significantly outperformed their BAU counterparts in all oral language and early 
literacy skill areas, with medium to large effect sizes. 

A TELL effect was noted for one of the distal measures (pre-post receptive and 
expressive vocabulary). No differences between the groups were found on other distal 
measures. 
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Preschoolers with Developmental Speech and/or Language Impairment: Efficacy of the Teaching 
Early Literacy and Language (TELL) Curriculum 

It is well established that preschool children’s early literacy and oral language skills are 
associated with subsequent reading achievement (Dickinson, McCabe, Anastasopoulos, Peisner- 
Feinberg, & Poe, 2003; Shanahan et al., 2008; Storch & Whitehurst, 2002). Early literacy skills, 
also referred to as code-focused skills, include conventions of print (e.g. directionality), 
beginning writing (e.g. name writing), grapheme knowledge (e.g. letter names), grapheme- 
phoneme correspondence (sounds made by each letter), and phonological awareness (e.g. 
beginning sound awareness). Code-related skills are among the strongest early literacy predictors 
of later reading decoding [National Early Literacy Panel Report (NELP), 2009]. In terms of 
language, evidence suggests that oral language skills, including vocabulary, language 
comprehension, and narratives, are associated with subsequent reading comprehension (for a 
review see Brinchmann, et al., 2017). However, oral language and early literacy skills also are 
interdependent and interactively predict some aspects of reading achievement including decoding 
and reading comprehension (Catts, Herrera, Nielsen, & Bridges, 2015; Lepola, Lynch, Kiuru, 
Laakkonen, & Niemi, 2016; Piasta, Groom, Khan, Skibbe, & Bowles, 2018). 
Risk and Young Children with Developmental Speech and/or Language Disorders 

Nearly all preschool programs in the United States provide some form of early literacy 
and oral language instruction as an essential component of preparing young children for the 
academic demands of formal schooling. A focus on these skill areas, while important for all 
young children, is of critical importance for children with development speech and/or language 
impairment (DSLI) to promote their readiness for school and access to the general curriculum. 


Young children with DSLI or developmental delay that includes DSLI represent the majority 
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(80%) of 3-5-yr-old children receiving education services through the Individuals with 
Disabilities Education Act in preschool classrooms (EDFacts Data Warehouse). Children with 
DSLI typically present with one of three profiles: Those with language impairment, those with 
speech impairment, and those with speech and language impairment. All groups of young 
children with DSLI may perform poorly on oral language and tasks when compared to their 
peers who are developing typically (Catts, Fey, Tomblin, & Zhang, 2002; Conti-Ramsden, St 
Claire, Pickles, & Durkin, 2012; Lewis et al., 2011; Peterson, Pennington, Shriberg, & Boada, 
2009; Schuele, 2004). 

Children with DSLI often experience challenges with reading fluency, reading 
comprehension, and other language-dependent academic tasks during the primary school grades 
and beyond (Catts, Herrera, Nielsen, & Bridges, 2015; Hayiou-Thomas, Carroll, Leavett, Hulme, 
& Snowling, 2017; Hulme, Nash, Gooch, Lervag, & Snowling, 2015; Snowling, Duff, Nash, & 
Hulme, 2016). The comorbidity of language impairment and reading disabilities ranges from 40- 
90%, (Catts, 2004) and the comorbidity of speech impairment and reading disabilities is from 25- 
50% (Peterson et al., 2009). Risk of reading disability for all children with DSLI persists, even 
when the speech and/or language impairment appears to have resolved (Zipoli & Merritt, 2017). 
When this elevated risk is considered, in addition to the fact that DSLI is a high incidence 
condition among young children receiving special education services, the importance of ensuring 
that these children are prepared for successful reading and schooling outcomes becomes a high 
priority. . 

Research indicates that language and early literacy skills are malleable (NELP, 2009; 
Diamond, Justice, Seigler, & Snyder, 2013) and instruction in these domains may lead to an 


increase in children’s school readiness (Hill, Gormley, & Adelstein, 2015; Magnuson, Ruhm, & 
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Waldfogel, 2007; Weiland & Yoshikawa, 2013; Yoshikawa, Weiland, & Brooks-Gunn, 2016). In 
a recent meta-analysis of early childhood education (ECE) programs, results showed that 
children who attended an ECE program where teachers provided explicit oral language and early 
literacy instruction demonstrated higher gains in these domains when compared to children who 
attended an ECE program that had a more general focus (Kholoptseva, 2016). Similar trends 
were identified in a recent review of early childhood oral language and early literacy programs 
(Chambers, Cheung, & Slavin, 2016). Although the benefits of preschool language and early 
literacy instruction are well documented, research indicates that the actual amount of oral 
language and early literacy instruction in preschool classes is limited, and often not well-timed or 
tuned to children’s interests and/or abilities (Guo, Justice, Kaderavek, & McGinty, 2012; Justice, 
Mashburn, Hamre, & Pianta, 2008; Pelatti, Piasta, Justice, & O’Connell, 2014). 

Over the past decade, education for young children with developmental disabilities 
increasingly is structured around a model of multi-tiered systems of support (MTSS), also 
referred to as response to intervention (Hebbeler & Spiker, 2016). In a MTSS system, instruction 
for all children begins with a whole class, tier one curriculum. Repeated progress measures 
monitor children’s learning, identifying those who need additional supports to acquire key skills, 
which may include tier two instruction (e.g., increase in practice opportunities) or tier three 
instruction (e.g., individualized interventions as indicated on an IEP). Central to the effectiveness 
of MTSS is provision of a tier one curriculum that has causal evidence for efficacy. This is 
crucial because in the application of MTSS, when children are not progressing within a tier one 
curriculum, teachers often decide to move them into a higher instructional tier. However, in the 
absence of causal efficacy evidence, it is possible that children’s lack of progress is due to use of 


a curriculum that is not efficacious rather than a need for higher tier of instruction. 
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Preschool Oral Language and Early Literacy Curriculum and Interventions for Children 
with DSLI: Overview of the Evidence Base 

The causal evidence base for tier one oral language and early literacy curricula is 
minimal. For example, the What Work’s Clearinghouse (WWC) intervention reports identify two 
tier one curricula, Literacy Express (Lonigan, Clancy-Menchetti, Phillips, McDowell, & Farver, 
2005) and Doors to Discovery™ , that showed positive effectives on children’s oral language 
and/or early literacy skills. Only Literacy Express showed effects on oral language and early 
literacy skills. Two supplemental literacy programs also showed positive effects for early literacy 
skills including DaisyQuest (online) and Headsprout Early Reading (software-based). Most of 
the early childhood curricula reviewed by the WWC had no effects on preschool children’s 
language or early literacy skills, a situation that creates significant challenges for implementation 
of a MTSS model. In addition, the majority of preschool language and literacy curriculum 
studies are with samples of young children who are at-risk due to economic disadvantage or other 
environmental factors. In contrast, there is little research conducted with samples of young children 
with developmental disabilities (Hebbeler & Spiker, 2016). Among early childhood curricula 
reviewed by the WWC, only one, Literacy Express, included children with developmental disabilities 
in the study sample, but findings did not include a separate analysis for these children, making it 
difficult to determine specific effects for this population. 

To date, most efforts to strengthen the oral language and early literacy skills of young 
children with developmental disabilities, and in particular those with DSLI, rely on interventions 
targeting specific oral language or early literacy components (e.g., vocabulary, phonological 
awareness, alphabet knowledge, print conventions) rather than comprehensive, tier one curricula. In 


the WWC two interventions, phonological awareness training and dialogic reading, are identified as 
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beneficial for young children with DSLI. Phonological awareness training includes several different 
strategies focused on improving young children’s ability to identify sounds in words independent of 
meaning.. When young children with DSLI receive phonological awareness training, either within 
their classrooms or individually, increases in their ability to identify sounds in words and segment 
words into sounds is documented (Kleeck, Gillam, & Mcfadden, 1998; O’Connor, O’Connor, 
Jenkins, Leicester, & Slocum, 1993). 

Dialogic reading is a set of strategies used by an adult during shared book reading to 
encourage children’s active participation. It is appropriate for use with an individual child or a small 
group of children. Research indicates that dialogic reading strategies significantly improve oral 
language skills for preschoolers with DSLI, including receptive and expressive vocabulary targeted 
during shared book reading (Towson, Gallagher, & Bingham, 2016), number and length of 
utterances (Crain-Thoreson & Dale, 1999), and correct responses to fact- and inference-based 
questions about the book (Whalon, Martinez, Shannon, Butcher, & Hanline, 2015). When 
comparisons were made between shared reading that did and did not include dialogic reading 
strategies, vocabulary learning and scores on an expressive language vocabulary test was higher for 
children when their adult readers used dialogic reading strategies (Hargrave & Sénéchal, 2000). 

Other targeted oral language and early literacy interventions not included in WWC reviews 
also appear efficacious when delivered to young children with DSLI. Studies document the 
effectiveness of shared book reading with embedded explicit print referencing (e.g., directionality of 
text, book cover, title, identifying letters and words) as an intervention that improves print concepts 
and alphabet knowledge (Justice, Logan, Kaderavek, & Dynia, 2015; Lovelace & Stewart, 2007). 
Shared story book reading is also an effective context for embedding language interventions to 


increase inferential language use (van Kleeck, Vander Woude, & Hammett, 2006). More recently, 
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digital storybooks with a narrator encouraging active child participation increased higher level 
vocabulary use for young children with DSLI (Kelley, Goldstein, Spencer, & Sherman, 2015). 

A substantial body of experimental and clinical research documents the efficacy of several 
oral language intervention techniques for young children with DSLI. Although referred to by 
different names including modeling, recasting, modeling with expansion, linguistic mapping, and 
focused stimulation, all provide an explicit language model that corresponds to a child’s focus of 
attention and includes the targeted linguistic form (e.g., vocabulary item, syntactic structure). In some 
techniques the language model includes a conversational request for a child to produce an utterance 
(e.g., “ball, can you say ball?”) while in others, the adult simply provides language models at a high 
rate, or includes a child utterance in a different or more complex syntactic form. These interventions 
are delivered to children in settings that range from more naturalistic (e.g., homes or preschool 
classrooms) to contrived (e.g., treatment at a clinic). Efficacy of these types of interventions is 
documented for increasing children’s receptive and expressive vocabulary, utterance length, use of 
grammatical morphemes (e.g., past tense), and syntactic structures (e.g., Buschmann et al., 2009; 
Camarata, Nelson, & Camarata, 1994a; Cleave et al., 2015; Fricke, Bowyer-Crane, Haley, Hulme, & 
Snowling, 2013; Girolametto, Pearce, & Weitzman, 1996; Johanne Paradis, 2015; Leonard, 
Camarata, & Camarata, 2008; McCartney, Boyle, Ellis, Bannatyne, & Turnbull, 2011; Roberts & 
Kaiser, 2011; Wilcox, Kouri, & Caswell, 1991) 

Several targeted interventions improve components of oral language and the early 
literacy abilities of young children with DSLI; however, the extent to which these interventions 
increase children’s readiness for school or subsequent reading success remains undetermined. It 


seems unlikely that interventions targeting isolated early literacy or oral language skills, by 
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themselves, offer young children with DSLI the opportunity to develop the complex array of 
early literacy and oral language skills that are important to early schooling success. 
Embedded Instruction and Effective Oral Language and Early Literacy Curricula 

Research indicates that children with and without disabilities demonstrate increased 
learning when oral language and early literacy instruction is embedded in classroom activities or 
a curriculum that offers sequenced learning opportunities (Horn, 2009; Horn, Lieber, Shouming, 
Sandall, & Schwartz, 1995; Kholoptseva, 2016; Neuman et al., 2015; Snyder et al., 2015; Terrell 
& Watson, 2018; Wilcox et al., 1991; Woods, Wilcox, Friedman, & Murch, 2011). Embedded 
instruction has a long and successful history in interventions delivered to children with 
disabilities and is a key component of high-quality learning opportunities for young children 
with disabilities in inclusive settings. 

An advantage to instruction embedded within a curriculum is the potential for providing 
young children with comprehensive learning opportunities to promote acquisition of a broad set 
oral language and early literacy skills. However, to realize the promise of using embedded 
instruction in this way, the curriculum must be efficacious, well-designed, and delivered with 
high fidelity. Efficacious curricula include those with causal evidence for effectiveness. Well- 
designed curricula are mapped to early learning standards and include a scope and sequence of 
developmentally appropriate activities and embedded instruction designed to teach children 
targeted skills (Hebbeler & Spiker, 2016; Kagan & Kauerz, 2006). High fidelity implementation 
is the extent to which a curriculum is delivered as intended. In addition, especially within an 
MTSS model, it is important that a curriculum include assessments to document children’s 
progress and to facilitate differentiated instruction. One published curriculum, Literacy Express, 


appears to adhere to all of these components and researchers included young children with 
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disabilities in their testing sample. The documented efficacy of Literacy Express suggests that it 
may be effective for young children with DSLI, but given the omission of separate data analyses 
for children with disabilities, the effectiveness for this population remains unknown. 

Although young children with DSLI acquire important language and early literacy skills 
in the same contexts as their peers with typical development, they require higher levels of 
explicit, embedded instruction to develop key language and early literacy skills that will promote 
their subsequent schooling success (Carta & Driscoll, 2013). An efficacious tier one curriculum 
with options for differentiated instruction can provide these higher levels of support, but to date, 
investigations of tier one curricula efficacy for young children with DSLI are very limited, 
establishing a clear need to examine curriculum efficacy for children with DSLI. 

Purpose of the Present Investigation 

The lack of efficacious curricula for young children with DSLI was the primary driver for 
our development and testing of a tier one, whole class, early literacy and oral language 
curriculum entitled Teaching Early Literacy and Language; TELL (Wilcox, Gray, Guimond, & 
Lafferty, 2011). The TELL curriculum promotes children’s growth in four early literacy skill areas 
(phonological awareness, alphabet knowledge, print conventions, writing) and three oral language 
skill areas (vocabulary, sentence length and complexity, and listening comprehension) that are robust 
predictors of subsequent reading fluency and reading comprehension. A key premise underlying 
TELL is that language and literacy goals can be taught and practiced in any preschool activity; thus, 
TELL was designed so language and early literacy learning opportunities are embedded within 
typical preschool activities, routines, and transitions as well as lessons with a primary focus on other 
learning areas (e.g., science, math). Further, given data documenting the efficacy of small group 


instruction for preschool children, including those with DSLI (Ball & Trammell, 2011; Connor, 
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Morrison, & Slominski, 2006), TELL includes small and whole group instruction. Overall, TELL is 
designed to increase oral language and early literacy learning opportunities within preschool 
classrooms through embedded explicit instruction, numerous incidental learning opportunities, lesson 
plans that are coordinated with effective language and early literacy instructional practices, and 
supports for ongoing assessment and differentiated instruction to meet children’s needs. 

Our first test of TELL efficacy was conducted with a small randomized controlled trial 
(RCT; Author et al, 2011) that included 118 children across 29 classes (19 TELL and 10 contrast). 
All children presented with DSLI and received special education in inclusive classrooms 
operated by local school districts. Although the study sample only included children with DSLI, 
all children within the class (those developing typically and those with disabilities that did not 
include DSLI) received the TELL curriculum. Outcome measures included two standardized 
tests of oral language and a benchmarked test of phonological processes and alphabet 
knowledge. Results indicated that in comparison to a business as usual (BAU) contrast group, 
children in TELL demonstrated greater gains on the phonological processing subtest of the 
Clinical Evaluation of Language Fundamentals — Preschool Second Edition (CELF-P2; Semel, 
Wiig, & Second, 2004), mean length of response on the Renfrew Bus Story (Cowley & Glasgow, 
1994), and letter sounds, beginning sound awareness, and rhyme awareness on the Phonological 
Awareness and Literacy Screening PreK (PALS-Prek; Invernizzi, Sullivan, Meier, Swank, & Jones, 
2004) . Children who received TELL also demonstrated greater gains on an investigator- 
developed receptive and expressive vocabulary test of the higher-level vocabulary targeted in 
TELL. There were no differences in performance between the TELL and BAU groups on the 


CELF-P2 core language, receptive language, or expressive language indices. 
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While promising, the causal evidence for TELL is limited, and replication with a larger 
sample of classrooms and children will contribute to an understanding of its efficacy for young 
children with DSLI. The primary purpose of the present investigation was to continue our 
investigation of the TELL curriculum with a larger-scale RCT and compare children receiving 
TELL with their peers who did not receive TELL. Further, we hoped to extend the results for 
TELL and young children with DSLI through development of curriculum-based measures 
(CBMs) to monitor growth of oral language and early literacy skills. We hypothesized that the 
effects of the smaller RCT would be replicated, and through our CBMs extended, by 
documenting a TELL effect for children’s oral language and early literacy skills growth over the 
preschool year. Specific research questions were as follows: 

1. Do young children with DSLI demonstrate increased growth in oral language and early 
literacy skills targeted in the TELL curriculum (as measured by curriculum-based 
measures; CBMs) when compared to peers with DSLI who did not receive the TELL 
curriculum? 

2. Do young children with DSLI demonstrate pre-post increases in distal oral language and 
early literacy skills, as measured by standardized or benchmarked oral language and early 
literacy instruments and a pre-post vocabulary test, when compared to peers with DSLI 
who did not receive the TELL curriculum? 

Method 

We implemented this RCT as a three-cohort design across a three-year period, with 
preschool teachers serving as the unit of assignment. After securing approval from our IRB and 
interested school districts, research staff met with preschool teachers to explain the purpose of 


the research and request their voluntary participation, including acceptance of random 
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assignment to implement TELL or to continue with their present curriculum. This process was 
repeated for each cohort and teachers who volunteered were stratified by school district and then 
randomized (via lottery) to the experimental (TELL curriculum) or business-as-usual contrast 
(BAU) condition. Some districts had all special education preschool classes on the same campus. 
When this occurred, there were TELL and BAU teachers on the same campus to ensure that we 
had both participant groups represented within each school district. We were minimally 
concerned with contamination because the TELL curriculum implementation requires 
specialized curriculum materials (books, lesson plans) and specialized training, neither of which 
were available for BAU teachers. We recruited child participants from consented teachers’ 
classes with the children assigned to TELL or BAU based on their teachers’ assignment. All 
teachers were paid stipends for their participation in the research. 
Sample 

Fifteen school districts formed the recruitment pool for teachers and young children with 
DSLI. We selected these districts because they all operated preschool programs through Part B 
of IDEA for young children with developmental disabilities, and included populations of 
children with DSLI. Although young children with DSLI may be enrolled in Head Start or other 
community preschool programs, most of these children are eligible for special education services 
through school districts. All preschool programs also enrolled tuition-paying neighborhood 
children with typical development and it should be noted that while we did not monitor progress 
for these children, they all received the TELL curriculum and their teachers monitored their 
learning as well as learning for children participating in our research. The sample for the present 
investigation included 318 preschool children with DSLI and their 98 teachers. Across the three 


cohorts of data collection nine teachers withdrew from the study (5 TELL and 4 BAU) for 
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personal reasons (e.g., moved, change in health status, pregnancy) or were they were let go by 
the school. A total of 29 children (17 TELL and 12 BAU) were lost to attrition because their 
teachers withdrew or their families moved. Hence a total of 91 teachers (42 TELL, 49 BAU) and 
289 children completed participation in all study activities. 

Teachers. Teachers were recruited from 58 schools across 15 school districts in a large 
metropolitan area. All teachers were female, 88% were Caucasian (not Hispanic), 6% were 
Hispanic, 2% were African-American/Black, and 3.3% were multiracial. One teacher did not 
provide information regarding her ethnicity/race. The teachers had a mean of 9.98 (SD=8.14) 
years of teaching experience at any grade and a mean of 5.97 (SD=5.22) years teaching young 
children with disabilities. All teachers were state-certified in either early childhood, early 
childhood special education, or special education, 50% had a bachelor’s degree and 50% had a 
master’s degree. 

All teachers had 10-16 preschool students enrolled in their morning and/or afternoon 
classes for a total of 140 class sessions. Forty-two teachers had one class daily, either morning or 
afternoon (16 TELL and 26 BAU), and 49 teachers (26 TELL and 23 BAU) had two classes per 
day with one group of children in a morning class and the other in an afternoon class. Class 
composition included tuition-paying peers with typical development (25-50%) and children with 
DSLI or other developmental disabilities (75 -50%) served through Part B of IDEA. Classes 
were staffed by the participating lead teachers and one teaching assistant and met for four or five 
half-days or full days per week. Occasionally a class had two teaching assistants (depending 
upon total class size and the complexity of the children’s needs). Additional related services 
were provided by physical and occupational therapists and speech-language pathologists (SLPs), 


as indicated on children’s individualized education plans (IEPs). 
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Children. All children qualified for IDEA Part B preschool services according to state 
criteria (which is used by all school districts in the state) and had a diagnosis of DSLI in the 
absence of an intellectual disability or other developmental or physical concerns. As noted 
earlier, DSLI includes children with speech or language disorders, and in our sample of 289 
children, 127 had speech impairment only, and 162 had language or speech and language 
impairment. The children’s eligibility for preschool services was determined by school district 
testing and all children demonstrated speech and/or language scores that were more than 1.5 SDs 
below the mean on a standardized test of speech and/or language. Additional inclusionary criteria 
for participation in the research were: (a) the children’s oral motor abilities were sufficient for 
speech, (b) their language skills included the ability to produce simple two- or three-word 
sentences, and (c) they had sufficient proficiency in English to allow reliable testing in English. 
English proficiency was determined by children’s teachers, confirmed by their parents, and 
verified by research personnel who conducted classroom observations, noting children’s 
preferred language for participation in preschool activities. The Kaufmann Assessment Battery 
for Children - 2nd Edition (K-ABCU; Kaufman & Kaufman, 2004) was administered to rule out 
intellectual disability (M=102.45, SD=12.06), with a score of 85 or greater required for inclusion 
in the research. No significant differences were noted between the TELL and BAU groups on 
KABC-II scores. 

The number of child participants ranged from two to four per class and included a total of 
202 males and 87 females ranging in age from 46 to 63 months (M=53.09, SD=3.61). The over- 
representation of males in the sample is consistent with data showing that males are more likely 
to have DSLI than females, and males are more likely to be enrolled in services than females 


(Black, Lindsey, Vahratian, & Hoffman, Howard, 2015; Boyle et al., 2011)The children were in 


Running head: EFFECT OF THE TELL CURRICULUM 14 


their pre-kindergarten year and eligible for kindergarten enrollment in the following school year. 
Parents reported 53.6% of the children being white (not of Hispanic descent), 2.1% as African 
American/Black, 25.3% as Hispanic, 12% as multi-racial, 1.7% as American Indian, and 3% as 
Asian. Eighty percent of children spoke English and it was the primary language spoken in their 
home; for the remaining 20%, English plus another language was spoken in their home. Table 1 
lists the range of maternal education and family income for each participant group. There were 
no significant differences in family income or maternal education between groups. 

All children received services from an SLP through their school districts. In addition, 54 
children (30 TELL, 24 BAU) received additional education or SLP services from private 
providers. The duration of the children’s intervention or participation in special education 
services prior to the investigation varied; some children had matriculated from the IDEA Part C 
system and others were identified at age three as eligible for IDEA Part B services. 

The TELL Curriculum 

TELL is a tier 1, whole-class curriculum that embeds incidental and explicit oral language 
and early literacy teaching practices within planned learning opportunities. The connection of 
evidence-based teaching/intervention practices to the curriculum is an important feature of TELL 
because it establishes a systematic link between efficacious oral language and early literacy 
component interventions/teaching practices and embeds them within a standards-based scope and 
sequence of instruction. The TELL practices are listed in Table 2 and include those known to 
support early literacy and oral language development and those identified as enabling acquisition 
of new skills through experimental and quasi-experimental methods and meta analyses (Cleave 
et al., 2015; Paradis, 2015; Literacy, 2008; Marulis & Neuman, 2010; Roberts & Kaiser, 2011; 


Wasik & Hindman, 2014). Supportive strategies (i.e., general supporting strategies), by themselves 
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do not promote skill acquisition, but they are viewed as necessary for acquisition of new oral 
language and early literacy skills (Smith, Warren, Yoder, & Feurer, 2004). In contrast, practices in 
the other groups shown in Table 2 enable acquisition of language and early literacy skills (Author et 
al., 1990; Author et al., 2011). 

TELL is theme-based and includes a scope and sequence of instruction mapped to early 
learning standards, materials (pictures, books, songs), and developmentally appropriate lesson plans 
that create language and early literacy learning opportunities. Given the importance of a diverse and 
rich vocabulary for children’s subsequent schooling and reading success (e.g., Beck & McKeown, 
2007), each theme is structured to provide children with multiple learning opportunities for academic 
(i.e., tier two) words. These words represent those that are not typically used during conversational 
interactions with young children outside of school, but contribute to building a diverse and rich 
vocabulary (e.g., schedule, creature, planet). 

In TELL, the oral language and early literacy learning opportunities are embedded within 
typical preschool activities (e.g., book reading, free play, dramatic play, music, art) and implemented 
with evidence-based incidental and explicit teaching practices (see Table 2). TELL objectives can 
also be embedded in lessons that target other important skills and developmental domains. For 
example, a science or math lesson may be organized to provide opportunities to learn target 
vocabulary or to promote listening comprehension. The links between the TELL scope and sequence, 
learning opportunities, and evidence-based teaching practices is by design and eliminates potential 
challenging situations where teachers are mastering teaching practices while also trying to create 
learning opportunities for use of that practice to teach specific skills. Hence, TELL structures the 


learning opportunities and teachers can focus on mastering the evidence-based practices. Figure 1 
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provides an overview of TELL and its components. TELL was described in detail in a previous 
article (Author, et al., 2011) and readers are referred to that source for additional information. 

TELL offers 34 weeks of instruction during a school year with 14 thematic units that are 
each two weeks in duration and review weeks that are planned approximately every fifth week. 
Review weeks vary somewhat to accommodate school vacation schedules. Just prior to a review 
week, teachers administer CBMs to track children’s progress on unit objectives. Teachers review the 
CBM data to select goals for the review week and plan differentiated instruction that includes small 
groups, adaptations to lesson plan expectations, strategies for scaffolding children’s responses, and a 
list of TELL materials or supplemental material. 

Twelve components structure learning opportunities to promote children’s growth in targeted 
language and early literacy skill areas. The components include a combination of materials (e.g., fact 
and ABC action cards, fiction and nonfiction books, table cards, social stories) and structured 
activities (e.g., book sequence, alphabet and phonological awareness games, writing, music and 
movement, and transitions with embedded language and early literacy learning opportunities). 
Component implementation occurs in whole group, story time, snack time, writing, and teacher-led 
activities. Table 3 illustrates the relationship between targeted oral language and early literacy skills 
and the TELL components that structure the learning opportunities. 

Implementation Supports 

Ongoing professional development was provided for teachers assigned to the TELL 
condition to ensure that they could establish and maintain a language and literacy-rich classroom 
environment and implement the curriculum with high fidelity. The methods for professional 
development followed those identified as effective in recent research about professional 


development (Cunningham, Etter, Platas, Wheeler, & Campbell, 2015; Domitrovich et al., 2009; 
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Dunst, 2015; Kraft & Blazar, 2017; McCollum, Hemmeter, & Hsieh, 2011; Pianta et al., 2017; 
Snyder et al., 2018) and included group training sessions and individualized, in-class coaching 
on a weekly basis in the fall and biweekly in the spring. An initial six-hour training session was 
held prior to the start of the school year to distribute the TELL curriculum and review all the 
components. Subsequently, there were 10, two-hour group sessions, with three in August and 
September and the remaining seven from October through April. The group sessions included a 
combination of brief didactic instruction regarding TELL teaching practices (see Table 4) 
followed by interactive learning opportunities to facilitate teachers’ TELL implementation in 
their classes (e.g., practice dialogic reading in small groups, using assessment to differentiate 
instruction, embedding language and literacy learning opportunities in art, music, movement, 
outside play, science, etc.). 

We assigned each teacher a coach from the research project staff. There were four 
coaches, three with a master’s degree in early childhood special education, and one with a 
master’s in reading. All coaches were certified teachers with extensive experience (>10 years) in 
teaching and/or coaching teachers in preschool classrooms serving young children with 
developmental disabilities. The coaches were randomly assigned (via lottery) across the classes, 
but stratified within each cohort to ensure that each coach provided support and coaching to 
teachers in more than one of the participating school districts. 

The coaching cycle included modeling, observation, feedback, reflection, and planning 
for the next visit. Some modeling and practice took place during the group sessions (e.g., 
identification of practices from a videotape, video of coach implementing the practice) to 
promote understanding of the practices being discussed, but most of the coaching took place in 


the teachers’ classrooms. The coaches conducted monthly fidelity checks (described in a 
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subsequent section) that informed coaching content. To introduce teachers to the TELL 
supportive and enabling instructional practices and their integration with required curriculum 
learning opportunity components, the coaches first modeled the practices in each teacher’s 
classroom, usually through co-teaching. Coaches observed the teachers as they implemented 
practices and the required TELL components (see Table 3) and then met with the teachers, 
usually immediately after their class sessions, to (a) facilitate teachers’ reflection and self- 
evaluations of the class session, (b) provide feedback about coaching observations, and (c) 
discuss needed changes for future implementation. 
Fidelity 

We obtained procedural fidelity (i.e., teachers’ adherence to the curriculum) in TELL 
classrooms monthly. We did this with a checklist of the required curriculum components listed in 
Table 3. Because the transition components each required different learning opportunities, we 
counted each as a separate component, for a total of 14 possible components. All coaches were 
trained to a reliable scoring level (>90% exact agreement) through observation of videotapes of 
several classrooms and comparison of their scores with a master score, before they conducted the 
first fidelity checks. To assess ongoing reliability of the procedural measure, a trained observer 
accompanied the coach and independently completed the checklist. We obtained these checks 
twice during the school year for each coach. Percentage of exact agreement was calculated for 
each observation and the mean agreement across the coaches was 98% (range of 91-100%). 

Given the schedule for TELL curriculum components (see notes in Table 3) it is 
challenging to observe all 14 components on a single day because some components are only 
required once (e.g., phonological awareness activity) or twice per week (e.g., fact cards). The 


coaches tried to schedule the monthly procedural checks on a day where all components could be 
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observed, but on occasion that was not possible. When a component could not be observed due 
to scheduling, coaches asked teachers if the activity occurred (or would occur) as indicated in 
their lesson plans and the component was scored as included or not, as per the teacher report. 
Fidelity scores were calculated for each teacher as the number of components that were 
implemented at each of the 10 observation points and then combined for a measure of procedural 
fidelity in the fall months (August through November), the winter months (December through 
February) and Spring months (March through May). Results indicated that procedural fidelity 
was high, with a mean across the teachers of 10.70 (SD=2.8) components implemented during 
fall months, 13.20 (SD=1.16) in winter months and 13.43 (SD 0.86) in the spring months. 
Business as Usual Contrast Group 

Teachers in the contrast classes continued with “business as usual” for curriculum and 
teaching content. TELL materials were not provided to the BAU teachers and we did not provide 
any additional professional development or implementation support. All BAU teachers received 
professional development through their respective school districts. Forty-seven percent of the 
BAU teachers reported using no curriculum. Others reported using Splash into PreK (2%), the 
Creative Curriculum (6%) or High Scope (1%). Remaining teachers (44%) identified their 
curricula as either “State Standards” or “Teaching Strategies Gold.” As confirmed through 
observations, none of the BAU classes used a curriculum that included a scope and sequence for 
teaching all of the early literacy and oral language skills that were targeted in TELL. Instruction 
in all BAU classes reflected state-approved early learning standards and the standards included 
oral language and early literacy benchmarks that corresponded to the skills targeted in TELL 
(e.g., letter names and sounds, phonological awareness, higher-level vocabulary, narratives, early 


writing, book handling, listening comprehension, print awareness). 
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We considered using the same procedural fidelity measure in the BAU classrooms that 
we used in the TELL classrooms. However, none of the BAU teachers used any of the TELL 
components (see Table 3 for a list of components) and a procedural fidelity score of zero was not 
useful for describing what language and early literacy teaching actually took place in the BAU 
classes. Hence, observations in the BAU classes focused on whether or not the teachers provided 
instruction that targeted a skill taught in TELL. 

The BAU teachers were observed by research assistants an average of 3.8 times over the 
school year. Four observations were planned, but the schedules for vacations, teacher 
professional development days, and teacher absences resulted in only three observations for 
some teachers. The research assistants were told that observations of the classrooms were 
required to understand language and literacy teaching in a variety of preschool special education 
classes. They were unaware of the specifics of the TELL curriculum including the materials, 
scope and sequence, and activities. Each observation lasted about two and one-half hours. All 
research assistants were initially trained to a reliable scoring level (>80% exact agreement) 
through observation of videotapes of several classrooms and comparison of their scores with a 
master score. 

Given that the skills associated with each TELL objective area are well-established as 
important foundations for subsequent schooling and reading success, and are targeted areas in the 
state early learning standards, we expected the BAU teachers to implement a variety of activities 
that targeted TELL objective areas. The observation data is summarized as the mean percentage 
of BAU teachers who implemented an activity that targeted a TELL objective (see Table 5). As 
indicated in the table, the majority of BAU teachers taught alphabet knowledge across the school 


year, followed by practices designed to increase sentence complexity and length (e.g. expanding 
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or recasting children’s utterances). During the fall term, less than half of the teachers targeted 
phonological awareness, print awareness, early writing, or vocabulary. The percentage of 
teachers targeting all oral language and early literacy areas increased in the winter observations, 
but either decreased or remained the same for the spring observations. 

Child Measures 

Relative to the TELL curriculum objectives, we conducted pre-post testing and then 
administered curriculum-based measures for skills that were explicitly taught in the TELL 
curriculum. All pre-post testing for the BAU and TELL classes was conducted by trained 
research assistants blinded to condition. These research assistants also administered the CBMs to 
children in the BAU classes. Because the CBMs for the TELL classes served a dual purpose of 
measuring growth and providing TELL teachers with data to differentiate instruction, the CBMs 
for the TELL classes were administered by the teachers (after training in administration from 
their coaches). Coaches observed CBMs several times for each teacher to ensure that there we 
administered and scored as planned. 

Pre-Post Measures. We administered four tests. One measure, the Clinical Evaluation of 
Language Fundamentals-Preschool 2" Edition (CELF-P2 Semel, Wiig, & Second, 2004), is a 
general, standardized test of language ability from which several different indices of oral 
language abilities may be obtained. We administered components to obtain standard scores for 
core language, expressive language, and receptive language. A second measure, the Test of 
Preschool Early Literacy (TOPEL, Lonigan, Wagner, Torgesen, & Rashette, 2007) was administered 
to obtain standard scores for the definitional vocabulary and the phonological processing subtest. Our 
third measure included a benchmarked assessment, the Phonological Awareness and Literacy 


Screening PreK (PALS-PreK, Invernizzi, Sullivan, Meier, Swank, & Jones, 2004). Four subtests were 


Running head: EFFECT OF THE TELL CURRICULUM 22 


given from the PALS-PreK including upper- and lower-case letter recognition, beginning sounds 
identification, and letter sound identification. 

Our fourth pre-post measure was a receptive and expressive one-word picture vocabulary 
test that assessed higher level receptive and expressive vocabulary targeted in TELL curriculum 
and many other preschool curricula. Administration procedures were identical to those for most 
standardized receptive and expressive vocabulary assessments. We first administered the 
expressive vocabulary test showing children a picture representing the targeted word and prompting 
them to produce the word (e.g., “Here is a furry animal, what do we call it?). We subsequently 
administered the receptive vocabulary test by showing the children an array of four pictures, with one 
being the targeted vocabulary word. Children were asked, to point to the picture corresponding to the 
word produced by the examiner (e.g., “Show me which one is a schedule”). To reduce the testing 
burden for the children, we developed two shorter tests rather than a single longer test, these 
were administered in August (pre) and December (post), and then again in January (pre) and 
May (post). The fall test was on vocabulary targeted in the curriculum from August to December 
and the spring test was for vocabulary targeted January to May. 

We developed and validated our picture vocabulary test in an earlier investigation of 
TELL (Wilcox et al., 2011). Specifically, we examined the concurrent validity between our 
picture vocabulary test and the CELF-P2 subtests that comprise the receptive and expressive 
language indices. Correlation with the CELF-P2 ranged from .31 to .47 for our receptive 
vocabulary test, and .50-.61 for our expressive vocabulary test (all p-values <.0001). 

Curriculum-Based Measures. These investigator-developed measures assessed oral 
language and early literacy skills targeted in TELL. We administered the CBMs to the children 


six times during the year with seven brief assessments per time (about 15 minutes total to 
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administer). All CBMs were scored as the number of correct items. None of the tests were timed. 
The first CBM set was administered approximately one month after curriculum implementation. 

Review weeks are built into TELL, and the CBMs are obtained just prior to a review week. This 

schedule for CBMs is a key aspect of TELL and provides teachers with information to plan both 
differentiated instruction and what to teach during the review weeks. 

We gave the CBMs to children in the BAU classes on the same schedule as that for 
children in TELL. It could be argued that the BAU children were not provided with an 
opportunity to learn skills assessed with the CBM. However, our observations of the BAU 
classes (see Table 5) confirmed that teachers provided instruction targeting these skills, including 
higher-level vocabulary. The vocabulary targeted in TELL included commonly taught tier two 
words and during observations in the BAU classes it was noted (anecdotally) that some of the 
TELL tier two words were included in books or lessons in the BAU classes. 

There were four early literacy CBMs including beginning sound awareness (8 items, 
children were shown a picture and asked to produce the sound the word starts with), print 
conventions (10 items, directionality, parts of book, author, etc.), letter recognition (26 items) 
and letter sounds (26 items). The same items were used for each of the six CBM assessments. 
The oral language CBMs included a story retell task to assess listening comprehension and a 
receptive and an expressive vocabulary test. For the story retell task, we scored children’s stories 
for inclusion of components of the original story for a maximum score of 8 points. We used a 
different story at each measurement point and each story included the same number and type of 
illustrations, used the same level of vocabulary words that were balanced for nouns and verbs, 


was the same length, and included the same syntactic structures. We piloted the different stories 
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with a small group of children developing typically and scores were similar across the children 
for different stories. 

The receptive and expressive vocabulary CBMs assessed the six academic (1.e., tier two) 
words targeted in themes just prior to administration of the CBMs. Hence, at each of the 
measurement points we assessed different words for a total of 36 different words assessed over 
the school year. This provided us a vocabulary assessment that was proximal (in time) to when 
the words were taught and is in contrast to the more distal pre-post vocabulary tests that assessed 
all of the higher-level words targeted in the TELL curriculum during the fall and then again 
during the winter/spring. 

Cronbach’s alphas for items on the CBMs indicated acceptable levels of internal 
consistency with values ranging from .83 to .99. Zero order correlations were calculated to 
determine the relation between our CBMs and other standard language and/or early literacy 
measures. The themed expressive and receptive vocabulary measures correlated significantly (r= 
A2 & .34, respectively, p <.01) with the CELF-P2 receptive language score, as well as the 
CELF-P2 expressive language score (7 = .40 and .36, p<.01). The listening comprehension 
CBMs correlated significantly with the CELF —P2 receptive and expressive language scores (7 = 
A3 & 47, p<.01) as well as definitional vocabulary subtest (7=.40, p<.01) of the TOPEL. The 
letter recognition, letter sounds, print conventions and initial sound awareness CBMs correlated 
significantly (7=.62, .73, .65, & .44, respectively, p<.01) with the corresponding PALS-PreK 
scales (i.e., alphabet recognition, letter sounds, print and word awareness, beginning sounds). 
Based on these results, we concluded that our CBMs were measuring similar underlying 


constructs. 
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Results 

In the following sections we discuss the analysis approach and findings for our pre-post 
child assessments (PALS-PreK, TOPEL, CELF-P2, Expressive and Receptive Vocabulary), 
children’s growth over the school year as measured by the early literacy and listening 
comprehension CBMs, and findings for the expressive and receptive vocabulary CBMs. Given 
substantial documentation of the relationship between maternal education and young children’s 
academic outcomes, cognitive development, and speech and language skills (e.g., Dollaghan et 
al., 1999; Harding, Morris, & Hughes, 2015; Magnuson, Sexton, Davis-kean, & Aletha, 2018), 
all analyses included maternal education as a covariate. In addition, we conducted a series of 
one-way ANOVAs to identify possible differences between the TELL and BAU groups for pre- 
test scores and while there were no significant differences, we conducted all child analyses with 
pre-test scores as a covariate. We also tested for differences in TELL program effects by cohort 
and found no significant differences. Hence, child data from each cohort were combined into a 
single sample to test the effect of TELL on all pre-post and growth measures. 
Growth Curve Modeling for Early Literacy and Listening Comprehension CBMs 

We administered the four early literacy CBMs and one listening comprehension CBM six 
times during the school year to the 289 child participants for a total of 1,445 potential 
measurements (5 CBMs x 289 children) for each time point. Data were missing for 10.7% of 
these measurement points and we assumed that these data were missing at random (MAR) for all 
analyses. We coded each of the six time points of CBM administration as weeks from baseline. 
We examined the multivariate distribution of the six repeated measures for each CBM with chi- 
square QQ plots and Mardia’s test for multivariate normality within each level of treatment. 
Based on the Mardia’s test, we rejected the null hypothesis of multivariate normality for each of 


the seven CBMs. We examined Chi-square QQ plots and Boxplots to assess the severity of the 
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departure from multivariate normality. The plots revealed a ceiling effect (i.e., right censored) 
for some of the CBM scores, particularly for the measurements taken at later time points. 

The measures were counts of number correct so we considered the square root 
transformation but had little success in producing a distribution closer to normal. Therefore, for 
variables with a ceiling effect, two models were fit, a linear mixed effects model with a normal 
distribution for the response variable and a linear mixed effects model with a right censored 
normal distribution for the response variable. We used Bartlett’s test to assess homogeneity of 
covariance matrices across the two treatment groups. We rejected the null hypothesis of 
homogeneity of variance for each of the five CBMs, due to ceiling effects. If appropriate, we fit 
models with heterogeneity across groups. Additional details about procedures for growth 
modeling arep included in the electronic supplement for this article. 

Table 6 shows growth parameter estimates for CBM models and Table 7 shows results 
for each CBM. There were significant main effects for time and condition, with all children 
improving in skill areas across the school year. There was a significant interaction of linear time 
x condition, indicating differences in the BAU and TELL groups at specific CBM time points, 
typically from scores at T3 through T6. In addition, the significant finding for time x time 
supports quadratic growth in either or both the BAU and TELL conditions for letter sounds, and 
listening comprehension. The time x time x condition effect for these five CBMs was not 
significant, indicating no significant differences between the groups in quadratic growth, or 
quadratic growth in only one of the participant groups. More specifically, for the letter names 
CBM, both the TELL and BAU groups demonstrated significant quadratic trends that were not 
significantly different from each other in that both groups trended toward a plateau, but children 


in TELL demonstrated greater magnitude and trended toward a plateau later than children in the 
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BAU group. For the listening comprehension CBM, children in the BAU group demonstrated 
quadratic growth, but children in the TELL group continued to improve while performance of 
children in the BAU group reached a plateau. 

CBMs for early literacy skills. Figures 2 through 4 illustrate results for the CBMs that 
assessed early literacy skills (i.e., letter recognition, letter names, print concepts, beginning 
sounds awareness). The figures include the marginal means and standard error bars for each 
CBM for TELL and BAU at each of the six time points. Approximate peaks of growth were 
estimated as weeks from baseline and for letter recognition and letter sounds, the peak of the 
growth curve was far outside the scope of the model for children in BAU and the TELL 
condition, indicating that all children were continuing to improve at the end of the school year. 
For the print concepts the peak of growth was outside of the model for children in TELL, but not 
for those in the BAU condition. 

Figure 2 shows the results for the /etter recognition and the letter sounds CBMs. For 
letter recognition there are significant differences in favor of TELL at T4 [¢ (90) = 1.36, p= 
0247], TS [t¢ (90) = 1.41, p = .0116] and T6 [¢ (90) = 1.5, p=.0057] with effect sizes (Cohen’s d) 
in the medium range at with effect sizes in the medium range at .37, .43, .51, respectively. For 
letter sounds CBMs, significant differences between the groups were noted at T3, [¢ (90) = 2.06, 
p = .0423], T4 [t (90) = 2.85, p = .0054], T5 [¢ (90) = 3.04, p = .0031] and T6 [¢ (90) = 3.15, p= 
.0022]. The effect sizes were in the medium range for T3 and T4 (d= .33 and .58) and were large 
for TS and T6 (d = .71 and .86). As shown in Figure 3, children who received TELL performed 
significantly better on the CBM for beginning sounds awareness at T4 [t (90) = 2.39, p = .0192], 
T5 [t (90) = 2.82, p = .006] and T6 [t (90) = 3.20, p = .0019]. Effect sizes were in the medium 


range at T4, T5, and T6 with d = .40, .52, and .65, respectively. Results for print concepts 
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(Figure 4) indicated significantly higher performance for children who received TELL at T3 [t 
(90) = 2.93, p = .0043], T4 [t (90) = 4.92, p<.0001], TS [¢ (90) = 5.60, p<.0001] and T6 [¢ (90) = 
6.08, p<.0001]. The effect sizes at T3 at .46 was in the medium range, and effect sizes for T4, TS 
and T6 ranged from large to very large (d= .78, .95 and 1.13, respectively). 

CBM for listening comprehension skills. Figure 5 shows the marginal means with 
standard error bars for listening comprehension. Children’s scores for listening comprehension 
were at the maximum value only 5.5% of the time, so results were obtained from a normal 
distribution model without censoring. As can be seen in Figure 5, there were significant 
differences in growth trajectories for both the TELL and BAU conditions. The growth curve for 
the BAU condition reached a peak at about 27 weeks, but the curve for the TELL condition had 
not started to plateau and was still increasing linearly at T6, meaning that children who received 
TELL were continuing to improve at T6 while the children in the BAU group reached a plateau. 
Children who received TELL scored significantly higher on the listening comprehension CBM at 
T2 [t (90) = 2.08, p = .040], T3 [¢ (90) = 3.32, p = .001], T4 [¢ (90) = 5.08, p < .0001], TS [¢ (90) 
= 5.65, p < .0001] and T6 [t (90) = 8.01, p < .0001] with effect sizes in the medium to very large 
range (d= .33, .50, .78, .92, 1.08, respectively). 

CBM Results for Targeted Expressive and Receptive Vocabulary 

As noted in the Method section, the receptive and expressive vocabulary CBMs were for 
different words at each time point, and we cannot assume that all words were of equal difficulty, 
rendering a growth analysis inappropriate. Table 8 includes the marginal means and standard 
error for each of these measures at each time point with the means indicating ceiling effects for 
receptive vocabulary and pronounced floor effects for expressive vocabulary. To examine the 


TELL effect for these two measures, a semi-parametric proportional hazard regression was used. 
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In this approach, the dependent variable is time to event, i.e., time from the start of the study 
until a child reaches the maximum score of six on a CBM administration. This method does not 
assume a particular distribution for the vocabulary scores or for the time to event. In this 
analysis, the TELL curriculum effect for receptive vocabulary was highly significant (p<0.0001). 
The estimated hazard ratio for the TELL effect was 2.83, meaning that the instantaneous rate of a 
success (where success is the event defined as achieving a score of 6) was nearly three times 
higher for children in the TELL group. 

To assess the TELL effect for the expressive vocabulary measures, we initially conducted 
the proportional hazards regression for time to event when a child first achieved the maximum 
score of 6. In this analysis, the TELL effect was significant (p<.0001), however, 50 out of 142 
children in the TELL group reached the criterion of six correct, but none of the 147 children in 
the BAU classes reached the criterion, so it was not possible to calculate the hazard ratio. 
Switching to a criterion of five correct, there was again a TELL effect (p=.0145). The estimated 
hazard ratio was 14.29, meaning that the instantaneous rate of success was higher in the TELL 
group when compared to the BAU group. 

Pre-Post Testing Results 

The mixed models for analysis of pre-posttests (i.e., CELF-P2, TOPEL, PALS-PreK, and 
the TELL Vocabulary) included a random effect for teacher and a fixed effect for time point 
instead of random intercept and slope terms for trend over time. We selected the unstructured 
(UN) model for R; to accommodate heterogeneity of variance. Given the selected covariance 
structure, we obtained the estimates for model fixed effects with estimated generalized least 
squares using SAS PROC MIXED software. Table 9 includes the least squares means, after 


accounting for maternal education and pre-test score, standard error, F’, and p-values for all pre- 
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post measures. After applying the false discovery rate (FDR) adjustment (Benjamini, Benjamini, 
& Yekutieli, 2001) to each tested construct, there were no significant TELL effects on any of 
subtests for the CELF-P2, the TOPEL or any of the PALS-PreK subtests (i.e., lower case letter 
recognition, beginning sounds identification, and letter sound identification). 

For assessment of the TELL academic vocabulary, we administered expressive and 
receptive vocabulary tests in the fall and again in the spring. The fall test was for words targeted 
in TELL themes August through December, and the spring test was for words targeted in TELL 
themes January through early May. Children’s scores during fall testing for both receptive and 
expressive vocabulary and during spring testing for expressive vocabulary were at the maximum 
possible value less than 5% of the time, so we conducted analyses (in a nested framework with 
pretest as a covariate) with a normal distribution without censoring. We obtained results for the 
spring receptive vocabulary testing from the normal distribution with a right-censoring model 
because 16.67% of the scores were at the maximum possible value. 

Table 10 shows the least squares means and standard error for the fall and spring 
vocabulary. After applying the FDR adjustment, results indicated a TELL effect, with children in 
the TELL condition outperforming their peers in the BAU condition. Findings for the fall 
expressive [F (1,276) = 50.71, p<.0001] and receptive [F (1,275) = 7.82, p=0.0055] vocabulary 
scores were significant, although the effect sizes were small with Cohen’s * f? at .18 and .02, 
respectively. Significant differences were also noted in the spring expressive [F' (1,244) = 80.72 
p<.0001] and receptive [F (1, 259) = 38.88, p<.0001] vocabulary scores with Cohen’s * f° effect 


sizes of .44 and .02, respectively (Selya, Rose, Dierker, Hedeker, & Mermelstein, 2012). 
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Discussion 

As we consider and interpret our findings, we do so with reference to each of our 
research questions, followed by a discussion of limitations and directions for continuing work in 
this area. 

Research Question One: Children’s Growth in Oral Language and Early Literacy Skills 

Results of the early literacy and listening comprehension CBM documented increased 
learning in all skill areas for children who received TELL with large to very large effect sizes by 
the final measurement point. Analysis of children’s acquisition of the tier two TELL receptive 
and expressive vocabulary also indicated increased expressive word learning for children 
receiving TELL, especially at later time points. Although we assessed different words at each 
time point, we think it likely that observed increases in children’s time to produce five words on 
an expressive vocabulary CBM was due to the TELL vocabulary teaching strategies, which are 
based on those found to be effective in language interventions focused on promoting word 
acquisition. 

Our observations in the BAU classes confirmed that children in those classes received 
instruction in oral language and early literacy skills targeted in TELL (e.g., Table 5). However, 
with the exception of alphabet knowledge, the percentage of teachers targeting TELL objectives 
for early literacy skills never exceeded 60%. A somewhat higher percentage (64%) of BAU 
teachers targeted oral language skills. Procedural fidelity measures indicated that TELL teachers 
increasingly implemented all TELL components, which collectively provided children with 
numerous language and early literacy learning opportunities. As noted in our introduction, if 
children with developmental disabilities, including those with DSLI, are to develop key language 


and early literacy skills prior to school entry, they require more intensive levels of support and 
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explicit instruction than do their peers with typical development. The fact that TELL teachers 
targeted language and early literacy skills in nearly all curriculum components across the school 
year suggests that as a group, the children with DSLI had more exposure to learning 
opportunities for the targeted skills. 

We hypothesize that differences in the children’s learning in the TELL and BAU classes, 
as reflected in the CBMs, are the result of four key ingredients that differentiate TELL from 
curricula or targeted interventions typically provided in BAU classes. First, TELL offers a 
specific scope and sequence for targeted skills so that children can build upon their knowledge as 
they transition from theme to theme across the school year. The scope of TELL is mapped to 
common state early learning standards and is also consistent with the Head Start Early Learning 
Outcomes Framework (2015). This pre-academic focus promotes a strong emphasis on oral 
language and early literacy activities, which improves these skills (Fuller, Bein, Bridges, Kim, & 
Rabe-Hesketh, 2017). The schedule for teaching TELL content was informed by the literature on 
language and literacy development, resulting in systematically sequenced content that provides a 
foundation for development of more complex skills. An explicit scope and sequence of learning 
objectives, as evidenced in TELL, is ‘an essential component’ of effective language and early 
literacy instruction (Bleses et al., 2018). 

A second TELL ingredient is the inclusion of whole and small group instruction. 
Typically, small group instruction is viewed as the second tier of multi-tiered systems of support 
(MTSS). However, in TELL a standard format for small group instruction is included within the 
Tier | curriculum. Given evidence for the effectiveness of small group instruction, especially for 


promoting language and early literacy skills (Connor et al., 2006; Gonzalez et al., 2014; NELP, 


Running head: EFFECT OF THE TELL CURRICULUM 33 


2009), we think it likely that this specific ingredient contributed to improved outcomes for 
children who received TELL. 

A third TELL ingredient is articulation with multi-tiered systems of support (MTSS) 
through the inclusion of CBMs. Intervention approaches for young children with developmental 
disabilities (including those with DSLI) increasingly rely on MTSS to facilitate development of 
pre-academic skills (e.g., language, early literacy, numeracy, social, behavioral) necessary for 
young children’s early school success (Fox, Carta, Strain, Dunlap, & Hemmeter, 2010; 
Greenwood et al., 2012; Hebbeler & Spiker, 2016). Central to the success of MTSS is ongoing 
assessment data to inform teachers about children’s progress. In TELL, the CBM assessment 
results enabled teachers to differentiate instructional objectives and identify children who may 
require additional tier two or tier three supports. With ongoing assessment data teachers can 
make decisions about content (e.g., increasing learning opportunities for specific skills) and 
teaching practices (e.g., explicit teaching practices, smaller group teaching) for specific children 
during planned review weeks and beyond. The CBM data also provides information to share 
with other interdisciplinary IEP team members (e.g., speech-language pathologists) thereby 
creating opportunities for collaborations to further increase children’s learning opportunities 
through tier three instruction. While we did not systematically track such sharing, many TELL 
teachers reported that the CBM data was shared with the children’s speech-language 
pathologists. 

The fourth key ingredient is the evidence-based, explicit teaching practices that are 
embedded during TELL learning activities (e.g., book reading, transitions), other pre-academic 
lessons (e.g., science, math), lessons designed to promote social-emotional competence, and 


classroom routines (handwashing, lunch, breakfast) and interest centers (e.g., writing, art, 
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dramatic play). Embedded instruction, sometimes referred to as naturalistic intervention, in 
combination with explicit teaching practices is a central aspect of learning opportunities that 
promote acquisition of skills for young children with DSLI. Research indicates that when young 
children with DSLI (and other developmental disabilities) have access to a sufficient number of 
high quality, embedded learning opportunities that include explicit instruction, they demonstrate 
increased learning of targeted skills (Carta & Driscoll, 2013; Kholoptseva, 2016; Rakap & 
Parlak-Rakap, 2011; Snyder et al., 2015). 
Research Question Two: Distal Effects 

Our second research question focused on performance of children who received TELL 
when compared to their BAU counterparts for the distal oral language and early literacy 
measures. As in our prior investigation (Wilcox et al., 2011) we found that children who received 
TELL outperformed their peers in BAU classes on the fall and spring pre-post receptive and 
expressive vocabulary testing. Children in the BAU classes likely had some exposure to these 
important Tier 2 words, but only children in TELL received explicit and embedded vocabulary 
instruction. We observed vocabulary learning by children in both conditions with very small 
between-group effect sizes for receptive and expressive vocabulary in the fall, and receptive 
vocabulary in the spring (Cohen’s * f?=.02, .18, .02, respectively). In contrast, the effect size for 
spring expressive vocabulary was in the medium range (Cohen’s * f?=.44) suggesting that over 
time expressive vocabulary learning was higher for children in TELL relative to their BAU 
peers. 

Results for the distal pre-post receptive vocabulary test were consistent with the 
receptive vocabulary CBMs, where all children demonstrated increases in receptive vocabulary 


at each testing point, but higher means were observed for children who received TELL. 
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Similarly, for children in the TELL group, results for the distal pre-post expressive vocabulary 
measure were consistent with the expressive vocabulary CBMs. In contrast, performance on the 
pre-post expressive vocabulary for children in the BAU group was inconsistent with the 
corresponding CBM. More specifically, children in the BAU condition produced many of the 
words on the distal expressive vocabulary test but did not produce these same words with the 
expressive vocabulary CBM. Our hypothesis is that children in the BAU classes likely learned 
the TELL vocabulary words incidentally and had limited exposure to explicit, embedded 
vocabulary instruction, particularly within the same timeframe as children in the TELL classes. 
Over time incidental exposure for children in the BAU classes was potentially sufficient for 
improved performance on a post-test, but perhaps insufficient the shorter timeframes associated 
with CBM testing. 

Our results for the distal standardized and benchmarked pre-post testing indicated no 
significant difference in performance between children in the TELL vs. the BAU classes. This is 
somewhat surprising given research suggesting that for young children, oral language and early 
literacy skills are highly interrelated and likely function as a single oral language construct (Catts 
et al., 2015; Dickinson et al., 2003; Foorman, Herrera, Petscher, Mitchell, & Truckenmiller, 
2015; LARRC, 2017). Hence, our proximal (CBMs) and distal (CELF-P2, TOPEL, PALS-PreK) 
assessments likely measured the same underlying language ability, but TELL effects were 
documented only with the CBMs. In addition, a lack of distal effects is inconsistent with our 
earlier, smaller efficacy trial of TELL (Wilcox et al., 2011) where effects were observed for two 
measures that were also used in the present investigation, the PALS-PreK letter sounds and 


beginning sounds subtests. 
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One explanation for observed differences between the present and our earlier 
investigation relates to the BAU or counterfactual conditions. In the time since our 2011 
investigation, federal and state requirements for most public preschool programs have changed 
and preschool programs are expected to use a curriculum, teach content that conforms to state 
early learning standards, and include oral language and early instructional activities (e.g., letter 
names and sounds, print concepts, sound awareness, narrative retells). In turn, early literacy 
instruction for the BAU classrooms in the present investigation likely had a more explicit focus 
on teaching early literacy skills and vocabulary than those programs that comprised the 
counterfactual in our earlier research. Observational results of the BAU classrooms (Table 5) 
document that BAU teachers provided vocabulary and early literacy instruction, particularly with 
alphabet knowledge and to a lesser degree, phonological and print awareness. Hence, an 
improved counterfactual condition may account for differences with our prior work. 

Our lack of findings for distal effects are consistent with other oral language and early 
literacy intervention or curriculum studies for young children with and without disabilities, using 
the same measures included in the present investigation and other, similar standardized measures 
(Kelley et al., 2015; Lonigan et al., 2011; Lonigan & Phillips, 2016; Mashburn, Justice, 
McGinty, & Slocum, 2016; Powell, Diamond, Burchinal, & Koehler, 2010). Although distal 
measures may be more stringent measures of treatment effectiveness, researchers consistently 
find a lack of effects attributable to oral language and early literacy interventions. It is worth 
noting that none of the distal measures in the present investigation, and many of those used in 
other investigations, were designed to measure growth of skills. The CELF-P2 is a diagnostic 
instrument that identifies children with language impairment and describes the functional areas 


of impairment. Similarly, the TOPEL identifies children at-risk for literacy problems, and given 
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that all children in the present investigation were at-risk, a lack of a TELL effect is not 
surprising. However, the apparent trend in the literature regarding distal effects creates a 
challenge, both for our work and that of others, when documenting efficacy of interventions or 
curricula. 

In addition to likely improvements in the counterfactual, there are other factors that may 
explain a lack of distal findings in the present investigation. One explanation pertains to 
differences in tasks used to assess constructs in the standardized tests and those used to practice 
skills representing those constructs in the TELL curriculum. For example, the phonological 
awareness subtest in the TOPEL is based on elision and word blending. For elision, a child is 
asked to say a word and then say what is left of the word after dropping specific sounds. Elision 
is not a skill targeted in TELL, nor is it one that is easily acquired without explicit instruction. 
For blending, a child is asked to listen to separate sounds and then combine them to form a word, 
while this skill was practiced in TELL it was within a supportive, meaningful context. For the 
CELF-P2, several tasks are used to arrive at a core language score, including following 
directions, recalling sentences, retrieving vocabulary words, and recall of dialogue from a story. 
In TELL, children practice all of these important language skills, but unlike the CELF-P2 tasks, 
oral language and early literacy skills are contextualized within shared book reading or other 
familiar activities. Generalization of skills from familiar routines and activities to a contrived 
situation is challenging for young children who are developing typically, and even more so for 
children with DSLI. It is possible that at the point of post-testing in our investigation, the 
children may not have stabilized their new oral language and early literacy skills to the degree 


where task and contextual factors no longer influenced their performance. 
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Another explanatory factor for a lack of distal effects is that standardized distal measures 
may not best suited for assessment of curriculum or intervention effects. Improvements for 
young children with DSLI are often incremental, and it takes a substantial amount of progress for 
a child to improve one standard deviation. Thus, a child may demonstrate significant progress as 
documented by CBM, but it may be insufficient for documenting improvement on a standardized 
instrument. As an alternative for children with disabilities, individual growth and development 
indicators (IGDIs) may be more sensitive to changes resulting from intervention (Mcconnell, 
Mcevoy, & Priest, 2002; Missall, Carta, McConnell, Walker, & Greenwood, 2008). Research 
with early literacy IGDIs, although not extensive, demonstrates the predictive validity of 
preschool early literacy IGDIs for end of kindergarten and end of first grade in oral reading 
fluency (Missall et al., 2007). In another investigation, early literacy IGDIs (picture naming, 
rhyming, and alliteration) were sensitive to longitudinal change and also reflected age-related 
differences in growth rates from ages three to four years (Roseth, Missall, & McConnell, 2012). 

Informal measures represent another viable alternative to standardized measures for 
young children with DSLI. In the present investigation we found TELL effects with pre-post 
vocabulary testing. This informal distal measure was aligned with the TELL vocabulary but not 
with tasks or contexts used to teach vocabulary in TELL or the BAU classes, suggesting that 
alignment of testing content may be more important than task differences. A large (and growing) 
body oral language and early literacy intervention research documents efficacious interventions 
with a variety of informal distal measures that directly align with anticipated intervention 
outcomes, but are removed from the intervention context (Camarata, Nelson, & Camarata, 
1994b; Dale, Crain-Thoreson, Notari-Syverson, & Cole, 1996; Girolametto et al., 1996; C. 


Greenwood et al., 2013; Kelley et al., 2015; Leonard et al., 2008; Towson et al., 2016; Wilcox et 
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al., 1991; Ziolkowski & Goldstein, 2008). Such informal measures may include (a) probes to 
elicit targeted language skills at specified intervals with different words or books, (b) analysis of 
spontaneous language samples that include use of grammatical morphemes, estimates of 
vocabulary size, or phonological analyses, or (c) documentation of targeted behavior that 
generalizes to another setting (e.g., child uses new vocabulary words at school and home). 

Informal distal measures, because they are one-step (or more) removed from the 
intervention context and tasks, potentially may suffice for documenting efficacy. Further, when 
informal measures are used in the context of well-designed experiments (e.g., RCT, quasi- 
experimental design, single case experimental research) and show positive effects of an 
intervention, efficacy is rarely questioned and often confirmed through meta-analyses or other 
similar procedures (Cleave et al., 2015; NELP, 2009; Marulis & Neuman, 2010; Roberts & 
Kaiser, 2011). 

There are various factors that may explain a lack of distal effects with the instruments 
used in the present investigation. However, an important question remains: Are standardized 
distal measures that are administered pre-post intervention necessary to demonstrate efficacy of 
an intervention or curriculum? We suggest that the answer is “no” if the distal measures do not 
align well with the instructional objectives. Further, we hypothesize that such alignment may be 
more critical for a suitable distal measure than task differences. If a distal measure does not 
directly assess anticipated learning outcomes of a curriculum or intervention, it seems unlikely 
that intervention/curriculum effects will be found and alternative distal measures are necessary. 
IGDI’s, well-designed informal assessments, or possibly proximal measures with sound 
psychometric properties all represent viable alternatives for determining efficacy. We do not 


suggest that standardized distal measures be omitted from efficacy research. Rather, we suggest 
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that alignment with anticipated intervention or curriculum outcomes is the central and key factor 
to consider when selecting distal instruments. Further, few, if any standardized measures of early 
literacy and oral language skills for preschool children align well with typical instructional or 
intervention objectives. Hence, at present, IGDIs, informal measures, and sound proximal 
measures may be the more appropriate way to determine causal effects of a given intervention or 
curriculum. 

Limitations 

We acknowledge limitations in this study in four areas. First, our category of children 
with DSLI included three different groups of children, those with language impairment, those 
with speech impairment, and those with speech and language impairment. As established in our 
introduction, each of these groups of children are at-risk for poor reading and early schooling 
outcomes, however, their specific strengths and needs may differ by group. We did not have the 
sample size to conduct fully powered analysis that considered these different groups, but this 
may be important to consider in future research to enhance our understanding of the impact of 
TELL on different groups of children. 

A second limitation is that CBM data was collected by TELL teachers, while the BAU 
data was collected by blinded research assistants. The CBMs provided immediate, important data 
for teachers to differentiate instruction and to understand each child’s skills. Although teachers 
were trained to administer the CBMs, and periodic checks were conducted for accuracy by 
research staff, teacher administration of these assessments may have introduced bias into the 
results. In future research, it will be important to have the CBMs obtained by assessors blinded to 
condition; but it is equally important to convey results to teachers as quickly as possible to 


facilitate their planning for differentiated instruction during review weeks. Potentially, the CBMs 
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could be administered with a tablet device that would generate graphs that are emailed to the 
teachers as soon as each child’s assessment is complete. 

A third limitation is the lack of a fidelity measure that was used for the TELL and the 
BAU classes. We assessed procedural fidelity only in the TELL classes, and although we 
observed the BAU classes to determine the extent to which TELL skills were targeted, we did 
not have the same measure for both conditions, although as mentioned earlier, if we had used the 
same fidelity measure the BAU teachers would have scored a zero at all observation points, 
because they did not implement any of the TELL components. Nonetheless, a lack of a common 
measure creates challenges when comparing TELL vs. BAU classes, and identification of a 
common measure is important to consider in future research with TELL, or other intervention 
efficacy studies. 

A final limitation concerns the supports that were provided for TELL implementation and 
the extent to which they were needed to promote children’s learning. We tested TELL as a 
package that included implementation supports. At this point it is unknown if gains documented 
for children who received TELL are due to the TELL curriculum, the implementation supports 
we provided (e.g., coaching and group sessions), or both. This is something that will be 
important to determine for future scalability. For example, if weekly coaching and an additional 
20 hours of group professional development are important to children’s learning, then it will be 
necessary to consider more cost effective ways to provide these supports because many 
preschool programs have limited funds and limited personnel resources. 
Conclusions and Future Research Directions 

A high-quality tier one curriculum is a necessary support to promote young children’s 


acquisition of language and early literacy skills that serve as a foundation for subsequent success 
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in reading fluency and comprehension. To date, research with designs that can infer causality is 
limited for tier one curricula that target these skills, with no prior research has examined tier one 
curriculum efficacy for specifically with young children with DSL. In the present RCT, we 
establish TELL efficacy for improving targeted oral language and early literacy skills for young 
children with DSLI. Despite limitations in our research, our work is among the first, larger-scale 
RCTs that demonstrate the efficacy of a comprehensive oral language and early literacy 
curriculum for young children with DSLI. When this finding is contextualized within the 
increasing use of MTSS, it suggests that a tier one curriculum, if implemented with high fidelity, 
can promote learning of essential pre-academic oral language and literacy skills for young 
children with DSLI. In addition, the inclusion of TELL CBMs informs differentiated instruction 
and/or the need for tier two or tier three instruction. The CBMs may also serve as a basis for 
collaboration with other disciplines who are involved in implementing IEPs. Overall, TELL 
demonstrates substantial promise for meeting oral language and early literacy needs of preschool 
children with DSLI. 

When thinking of future research, an area meriting investigative attention pertains to 
longer-term impacts of TELL and other preschool curricula. Increases in children’s oral language 
and early literacy skills during preschool, while predictive of later reading success, may or may 
not lead to improved school success and reading outcomes for young children with DSLI. In fact, 
research indicates that in many cases preschool program effects fade rapidly upon kindergarten 
entry and often disappear by the end of first or second grade grade (Hill et al., 2015; Magnuson 
& Duncan, 2016; Puma, Bell, Cook, & Heil, 2010; Wake et al., 2015). In contrast, others have 
found more lasting benefits, both for the overall preschool experience (e.g., Claessens & Garrett, 


2014) and targeted early literacy interventions (Justice, Logan, & Kaderavek, 2017). TELL falls 
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somewhere in-between a comprehensive curriculum and targeted targeted interventions, and 
children who receive TELL or a similar progrtam may demonstrate lasting gains, but future 
research is needed to establish such a link. 

Another issue for future research pertains to TELL and other curricula efficacy with more 
diverse populations. To date, TELL has only been tested with populations of young children with 
DSLI. However, TELL is a whole class curriculum, and because we tested efficacy in inclusive 
classrooms (e.g., enrolled children with and without disabilities) all children received the 
curriculum. Although anecdotal teacher reports support the promise of TELL for all children 
who were in the classroom, the efficacy of TELL for young children who do not have 
disabilities, and are not otherwise at risk has not been established. At present we (Authors, 2017) 
are conducting an RCT of TELL for young children with typical development enrolled in Head 
Start, or other community preschools serving children who are economically disadvantaged. 
Although preschool teachers in these programs typically are not certified teachers, and have 
lower levels of educational attainment, with implementation support they are able to achieve 
high fidelity to TELL. In terms of children, data from two cohorts of participants indicates strong 
TELL effects when children who receive TELL are compared to their peers in BAU classrooms. 
Children from economically disadvantaged homes are often viewed as at-risk, albeit the reasons 
are different from risk for young children with DSLI. Hence, it would also be useful to test the 
curriculum in early care and education programs that enroll children with more economic 
diversity, and who likely have more advanced oral language and early literacy skills than the 


populations we have studied to date. 
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Table 1 


Family Income and Maternal Education by Participant Group 


Variable 


TELL (n =142) 


Contrast (n =147) 


Annual Household Income 
Less than 10,399 
10,400-13,999 
14,000-17,599 
17,600-21,199 
21,200-24,799 
24,800-28,399 
28,400-35,599 
35,600-64,999 
65,000-99,999 

100,000 or more 
Missing/Response not given 
Maternal Education 

Some grade school 

Some high school 

High school graduate 

Some college, no degree (1-3 yrs) 
Associate degree in college (2 
yrs) 

Bachelor’s degree 

Master’s degree 

Ph.D. 


Missing/Response not given 


n(%) n(%) 
11 (7.7) 8 (5.4) 
1 (0.7) 4 (2.7) 

3 (2.1) 5 (3.4) 

1 (0.7) 7 (4.8) 

4 (2.8) 3 (2.0) 

3 (2.1) 5 (3.4) 

8 (5.6) 8 (5.4) 
29 (20.4) 34 (23.1) 
21 (14.8) 22 (15.0) 
21 (14.8) 18 (12.2) 
40 (28.2) 33 (22.4) 
: 1 (0.7) 

8 (5.6) 10 (6.8) 
19 (13.4) 21 (14.3) 
51 (35.9) 38 (25.9) 
18 (12.7) 22 (15.0) 
36 (25.4) 39 (26.5) 
9 (6.3) 12 (8.2) 

1 (0.7) 3 (2.0) 

: 1 (0.7) 
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Table 2 


TELL Supportive and Explicit Teaching Practices 


General Supporting Strategies 
1. 


Uses strategies to help facilitate 
processing and comprehension (e.g., slows 
rate of speech, using visual aids) 

Rephrase questions/comments to simplify 
when child appears not to understand 
Engages child prior to talking or providing 
instructions 

Provides children with a sufficient amount 
of time to respond to questions and 
directives (minimum of 5 seconds) 
Provides children with choices or options 
when he/she doesn’t respond to a question 
(e.g., provides a two-option choice) 


Print Concepts and Alphabet Knowledge 


1. 


2 


3: 


Points to letters/words and tracks print in 
books 

Asks children to find letters or words on a 
page or in the environment 

Discusses parts of a book (cover, spine, 
author, illustrator, where to begin reading 
on a page, etc.) 

Points out print in the environment 

Says letter names and asks children letter 
names 

Says letter sounds and asks children about 
sounds that letters make 


Language Teaching Practices 


Phonological Awareness 


1. Uses words to describe the environment 1. Explains concept of rhyme (e.g., 
and relationship among items “rhyming words are words that sound 

2. Uses dialogic reading during story time alike or sound the same at the end of the 

3. Demonstrates concepts with appropriate word”). 
vocabulary and props 2. Identifies rhyming words in context 

4. Uses SEER (see, example, explain, repeat) | 3. Asks children questions about rhyming 
methods to teach vocabulary words (e.g., “Which words rhyme?” 

5. Uses linguistic mapping strategies “What word rhymes with _—?””) 
(produces sentences that includes 4. Asks children to identify words/objects 
vocabulary or states the meaning beginning with an initial sound or letter 
associated with a child’s ongoing activity) (e.g., “Which one starts with the letter S 

6. Recasting and Expansions: Verbally or with the sound /ssssss??’’) 
recasts children’s utterances to model 5. Claps syllables in words, Counts syllables 
more complex grammar and higher level in words, Asks children to count syllables 
vocabulary in words 

7. Modeling complex vocabulary and elicit 6. Segments words by sounds (e.g., “c-a-t, 
more language with explicit prompts with what word is that?’’) 
either a direct request (e.g., “Here is a 
XXX can you all say XXX?) or 
conversational prompt to extend language 
(e.g., Yes, this is a fast truck, what else 
can you tell me about it?). 

Writing 

1. Models writing and letter formation 
Provides children with opportunities to form, trace, or write letters on paper, chalkboard, 
etc. 

3. Takes dictation during a activities 

4. Asks children to write (drawing acceptable) names during activities 
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Table 3 
Required Curriculum Learning Opportunity Components x Activity x Targeted Skill Sets. 
Curriculum Learning Targeted Skill Area 
Opportunity Component Activity PA PC AK Wr Vocab SLC 
1. @ABC warm-up (letter Teacher-led (small group); Xx Xx 
names) followed by PA _ Story time (small group); 
activity Transition (whole group) 
2. Focus book reading Story time (small group) x x 
3. Focus book sequence Story Time (small group) 
e Picture walk x x 
e Dialogic reading x x 
e Read it again x x 
e Story re-tell xX x 
4. Vocab teaching sequence Whole group x x 
(say word, explain p Options: Dramatic Play, 
meaning, give example, — Science, Math, Art 
repeat word) 
5. Table tent cards (script Snack x x x 
on back) 
6. “Fact cards (expository, Whole group; story time; x x 
theme-related) snack 
Options (dramatic play, 
science, math, art) 
7. Social story cards Whole group x 
8. ABC action cards (letter | Whole group x x x 
chants) 
9. Music and movement Whole group x x 
10. Three transitions (one Moving from one activity to X x x 
each for vocab, alphabet —_ another (activities variable) 
knowledge, phonological 
awareness) 
11. Writing (drawing, letters, Writing center xX XxX 
dictation to teacher) 
12. PA warm-up (sound Whole group x 


patterns) 


Note. PA=phonological awareness, PC=print concepts, AK = alphabet knowledge, Wr=interactive writing, 
Vocab=higher level vocabulary, SLC=sentence length and complexity. The components were required daily, unless 
otherwise noted. 

* Required once per week per child, could be implemented daily for different groups of children or once per week 
with different groups rotating through while the rest of the children were engaged in another activity 

> Options for activity blocks varied across teachers, allowing customization of the curriculum to meet their 
preferences 

° Required twice weekly 
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Table 4. 


Overview of Professional Development & Coaching 


When Topic for Group PD Topic for Coaching 
(number of coaching sessions) 
July e Introduction to TELL and review of | NA 
components and implementation 
August Curriculum-based assessments and | NA 
using them to differentiate 
instruction 
August Setting up the classroom (2) Creating a language and literacy rich 
environment environment, use of general supporting 
General supporting practices practices to engage children and promote 
interaction with each other 
September Explicit and incidental language (3) Embedding language teaching practices 
teaching practices in other activities (e.g., art, music, outdoor 
play, etc.) 
October Dialogic reading and teaching (3) Embedding explicit and incidental 
vocabulary teaching practices to promote vocabulary, 
Print concepts and alphabet print concepts, and alphabet knowledge 
knowledge teaching during small group book reading and other 
activities (e.g., dramatic play, art, music, 
etc.) 
November Differentiating instruction based on | (2) Differentiated instruction within TELL 
the CBMs components 
December Implementation issues and (3) Embedded learning opportunities that 
solutions are differentiated for individual children’s 
needs 
January Phonological awareness teaching (3) Embedded, explicit instruction to 
practices promote children’s phonological 
Teaching early writing awareness, interactive writing, 
TELL in action video review and 
sharing implementation strategies 
February TELL in action video review and (3) 
sharing implementation strategies 
March TELL in action video review and (2) 
sharing implementation strategies 
April TELL in action video review and (3) 


sharing implementation strategies 
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Table 5. 


Average Percent of BAU (n=49) Teachers who Implemented Activities with Learning 
Opportunities for a TELL Objective x Time of Year. 


Fall Winter Spring Total for the 
School Year 
eink cake of different 59 35 93 187 
Teaching Alphabet Knowledge 74.32 85.25 86.94 82.64 
Teaching Phonological Awareness 40.04 59.72 57.69 53.01 
Print Awareness 43.75 63.62 66.97 59.00 
Interactive Writing 37.48 59.1 58.69 52.64 
| Teaching Target Vocabulary | 4148 | 72.99 | 65.08 | 6037 | 
Sentence complexity 58.84 74.47 69.34 67.72 


Note. Fall included August-October; Winter included November through January; Spring 
included February-May. For teachers with morning and afternoon class sessions scores were 
collapsed across the two classes and it was treated as a single observation. 
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Table 6. 
Growth Curve Estimates for the TELL and BAU Groups 
Variable Bo By Be Bs 
TELL BAU TELL | BAU TELL | BAU TELL BAU 
Letter 11.08" | 10.43°™" | 1.53" " | 1.53" | 1.61°" | 1.20°" | -0.034 | -0.034 
Recognition 
Letter Sounds | 3.98" 481° | 1.25" | 1.25°™ | 1.74" | 0.99" | -0.029 _| -0.016 
Beginning 1.277" | 1.63°" | 0.57" | 0.57°" | 0.66" | 0.39" | -0.006 | -0.006 
Sounds 


Listening 


3.14" 


2.40°" 


0.123" 


0.123" 


oR 


0.35 


0.44°™ 


-0.001 


Comprehension 


Print Concepts | 4.72" | 4.69"" | 0.13 0.13 0.61" | 0.36" | -0.015 | -0.015 


-0.036"" 


Note. Estimates are for the model yj jx. = Bo + Bi MEjcx1y + Pot + Bs t2 + Ux) + Eijer, Where t 


= time and ME = Mother’s Education. 


*p<.01; ** p<.001; *** p<.0001 
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Table 7. 
Growth Curve Results for Curriculum-Based Measures 
Early Literacy Skills Oral Language 
Letter Letter SoundID _ Beginning Print Concepts Listening Comprehension 
Recognition Sounds (Story Re-Tell) 
Effect df t df F df t df t df F 
Maternal 90 3.43°° 1,273 14.78°" 90 415° 90 1.50 1,280 2.22 
Education 
C 90 0.44 1,94.8  5.59° 90 -0.84 90 -0.07 1,82.3 18.91" 
T 90 419°" 1,75 483.63" 90 3.79" 90 3.36" 1,1284 161.3°" 
T*T 90 -1.44 1,1029 6.18" 90 -0.57 90 -1.51 1,1268 12.65" 
T*C 90 2.69" 1,275 50.41" 90 4.77 90 5.07" 1,1279 39.14" 
T2sC 90 -0.06 1,687 0.16 90 0.74 90 -0.56 1,846 2.89 


Note: C=Condition, T=Time; *p<.01; **p<.001; ***p<.0001 
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Table 8 


Estimated Marginal Means (Standard Error) for Receptive and Expressive Themed Vocabulary 
Testing x Group x Measurement Point 


Average Month Receptive Vocabulary Expressive Vocabulary 
Tested TELL BAU TELL BAU 
1 4.84 3.16 1.56 .09 
(.23) (.19) (5.80) (8.11) 
2 S17 a9 1.85 42 
(.21) (.16) (6.68) (5.55) 
3:5 6.46 4.52 2.26 85 
(.21) (.16) (2.53) (2.87) 
6 7.43 Si 2.67 1.26 
(.21) (.17) (11.47) (1.53) 
73 7.62 5.30 219 1.36 
(.22) (.17) (18.18) (2:35) 
8.8 7.65 5.18 2.75 1.38 
(.26) (.21) (27.26) (4.62) 


Note. The words tested at each measurement point were six, tier 2 words that were taught in the 
preceding two or three thematic units. 
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Table 9 


Least Squares Means, (SE) and Statistical Details for CELF-P2, TOPEL, and PALS-PreK x 
Group Post-Test Scores 


Test Subtest BAU TELL F-value —_— p-value FDR 
(SE) (SE) df=1,421 p-value 
CELF-P2 Core Language 89.52 90.27 0.36 0.5495 0.6182 
Standard (0.86) (.91) 
Scores Receptive 93.11 94.40 0.7 0.4035 0.5943 
Language (1.06) (1.12) 
Expressive 84.98 87.86 6.08 0.0140 0.0630 
Language (0.80) (0.85) 
TOPEL Definitional 96.16 96.77 0.71 0.7067 0.7067 
Standard Vocabulary (1.11) (1.18) 
Scores Phonological 91.16 92.74 0.42 0.4168 0.5943 
Awareness (1.33) (1.41) 
PALS- Upper Case Letter 7.25 fae 0.29 0.2945 0.5943 
PreK Recognition (0.31) (0.33) 
Number Lower Case Letter 23.05 22.60 0.41 0.4118 0.5943 
Correct Recognition (0.37) (0.37) 
Letter Names 18.34 iWmew, 0.46 0.4622 0.5943 
(0.73) (0.74) 
Beginning Sound =: 17.67 20.15 8.43 0.0093 0.0630 
Awareness (0.58) (0.62) 


Note: Reported least squares means are those after accounting for pretest score and maternal 
eduction; CELF-P2=Clinical Evaluation of Language Fundamentals Preschool 2" Edition; 
PALS-PreK = Phonological Awareness and Literacy Screening Preschool; TOPEL = Test of 
Preschool Early Literacy. 
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Table 10. 


Least Squares Means (SE) for Pre-Post Fall and Spring Vocabulary Tests 


Pre-Test Post-Test 

Test Subtest BAU TELL BAU TELL 
Fall Receptive 29.88 31.12 33.01 35.73 
Vocabulary (0.55) (0.56) (0.52) (0.49) 
Expeaive 11.34 12.39 14.47 18.27 

(0.48) (0.51) (0.54) (0.56) 

Spring Receptive 30.97 31.03 33.49 36.75 
Vocabulary (0.60) (0.59) (0.54) (0.45) 
Pxpreseive 14.41 14.37 18.44 23.15 

(0.57) (0.54) (0.63) (0.68) 


Note. We report least squares means after accounting for maternal education. 


Figure 1. 


Overview of TELL Curriculum, Curriculum-Based Measures, Components, and Implementation Supports 


Implementation Supports 
e Group Content Training 


Sessions (26 hours 
spread over 11 sessions 
during the school year) 

e |n-Class Coaching 
(weekly fall & biweekly 
spring) 

e Coaching Cycle (observe, 
model, performance 
feedback) 

e Weekly Reflective 


Thematic 

14 Themes 
(each 2 weeks in 

duration) 


Discussion 


Evidence-Based Language 
and Early Literacy Teaching 
Practices 

e General Supporting 
Strategies 

e Explicit Language 
Teaching Strategies 

e Alphabet and Print 
Concepts 

e Phonological Awareness 

e Emergent Writing 


Scope & Sequence of 
Instruction 


e Vocabulary 

e Language 
Comprehension & 
Complexity 

e Phonological 
Awareness 

e Alphabet Knowledge 

e Print Knowledge 

e Early Writing 


ae 
a Plans / 


Integrations of Oral 
Language and Early 


Literacy Skills 
e Art Lessons 


e Math Lessons 

e Science Lessons 

e Fine Motor Activities 

e Social Emotional 
Growth 


Scripted Teachin 
Activities for 
Required Learnin 
Blocks for each 
Theme 
e Circle 
e Small Group 
(Storytime, 
Phonological 
Awareness) 
e Early Writing 
e Transitions 
e Snack 
e Integrations 


a 


ROE 


Instruction 


Lesson 
Materials 


ee 


Suggested Teaching 
Activities 
e Playtime 
e Fine motor 
e Dramatic Play 
e Outdoor Play 


e Teacher's Manual 

e Focus Books (fiction 
& nonfiction) 

e Fact Chat Cards 

e Social Stories 

e Music 

e ABC Action Cards 

e Transition Cards 

e Phonological 
Awareness Cards 

e Canvas Bag Storage 
System (color coded 
for organizing 
materials for each 
theme) 


Reviews every 5" week 
(informed by CBMs) 


eLetter names 
eLetter sounds 
e Phonological Awareness 


(beginning sounds of words) 


e Receptive & Expressive 
Vocabulary 

e Print Concepts 

e Story Retell (listening 
comprehension and 
expressive language) 


Figure 2. 
BAU and TELL Letter Recognition & Letter Sounds (Lrecog, Lsnds) Estimated Marginal Means, Standard Error Bars and Significant 
Differences x Average Month of School Year CBM was Administered 
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Figure 3. 


Beginning Sounds Awareness Estimated Marginal Means, Standard Error Bars and Significant Differences x Average Month of 
School Year CBM was Administered 
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Figure 4. 


Print Concepts Marginal Means, Standard Error Bars and Significant Differences x Average Month of School Year CBM was 
Administered 
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Figure 5. 


Listening Comprehension Estimated Marginal Means, Standard Error Bars and Significant Differences x Average Month of School 
Year the CBM was Administered 
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ELECTRONIC SUPPLEMENT 

Model for Normal Distribution 

A linear mixed model using a normal distribution without censoring for the response 
variable was used with fixed effects factors for treatment and time, random effects for teachers, 
and random coefficients at the child level. Child is crossed with time, but children were nested 
within teachers, and teachers were nested within treatment level. Teachers had only one or two 
classrooms in the study and an average of three children per class; thus, there were too few 
children per class and classrooms per teacher to consider an effect for classrooms nested within 
teachers. Similarly, school nesting was not included in the model because there were not enough 
classrooms per school to estimate the effect of school after classroom effect was in the model. 
The design is essentially a split plot with an added random factor for teachers. The treatment 
factor had two levels, TELL vs BAU. The mixed model specified random coefficients for 
intercept and linear slope, so each child had his or her own growth trajectory, and the random 
intercept and slope terms accommodate heterogeneous variance and covariance across time. 
Because the random intercept and slope terms accommodate correlation and heterogeneity of 
variance across time, the independence structure o71 was selected for the covariance matrix of 
the random error terms for the six measurements on child 7. Variance components were estimated 
by the method of restricted maximum likelihood. We used the Kenward-Rogers (KR) method to 
calculate approximate degrees of freedom for F tests of model fixed effects. The likelihood ratio 
test and the Bayes Information Criterion (BIC) were used to compare models with heterogeneity 
of variance across treatment groups. LOESS curves were examined to suggest the degree of the 
polynomial to represent trends across time prior to estimation of the fixed effects for the growth 


curve model. The model fixed effects were estimated by the method of generalized least squares 


using SAS PROC MIXED. This model was used for analysis of two CBMs, listening 
comprehension and beginning sounds identification. 
Model for Right/Left-Censored Distribution 

There were ceiling effects for three of the early literacy CBM (print concepts, letter 
recognition, letter names) with scores at the maximum for 10.4%, 21.25% and 24% respectively. 
In addition, scores on the receptive vocabulary CBM were highly skewed, with 52% of the 
children scoring at the maximum of six correct overall and 88% scoring at the maximum at the 
final measurement point. In contrast, 29% of scores for the expressive vocabulary CBM were at 
the minimum value of 0 correct and only 6% were at the maximum value of 6 correct. For each 
of these CBM, results were obtained a linear mixed effects model with a right (or left) -censored 
normal distribution for the response variable (Vock, Davidian, Tsiatis, & Muir, 2012). This 
approach is sometimes known as a Tobit model. Y;;,; would have been observed if there had 
been no censoring. However, due to censoring, we observe Q;;,,, which takes on the value Y; jx 
for Yijx1 < Uj; and takes on the value l;;, the known upper limit, otherwise. For each CBM, the 
known upper limit /;; was the same for all time points and treatment levels. Variance 
components and fixed effects parameters were estimated by the method of maximum likelihood. 
Parameter estimates from the linear mixed effects model without right censoring were used as 
start values for maximum likelihood estimation of the right censored model. 

The right (left)-censored model was estimated with SAS PROC NLMIXED, which 
presents some limitations compared to SAS PROC MIXED. The primary limitation is 
specification of a maximum of two random components, necessitating a choice between random 
effect at the child or the teacher level. Estimation of the right (left)-censored model with random 


effect at the child level encountered substantial difficulties with convergence, and the right-(left) 


censored model with both random intercept and random slope rarely converged. Therefore the 
right (left)-censored models were estimated with random teacher effect. 

Because the structure of the covariance matrix for the six measurements across time may 
have been misspecified in the right-censored model, the robust “sandwich” estimator (White, 
1980; Liang and Zeger, 1986) was used for estimated standard errors and t tests of fixed-effect 
parameters. Comparing results from the SAS PROC MIXED model and the right censored- 
model, the same model fixed effects were found significant in both approaches. Usually the 
TELL effect was larger in magnitude under the right-censored model, which is intuitively 
reasonable since TELL children tend to reach the maximum possible score at an earlier time 
point. Hypothesis tests on fixed effects parameters were conducted using Wald statistics. 
Whereas the Kenward-Rogers denominator degrees of freedom from SAS PROC MIXED are 
over 900, the degrees of freedom for t-tests (square root of Wald statistics) are obtained from the 
number of teachers and were usually equal to 57. Degrees of freedom equal to 57 is large enough 


so that the distribution of the test statistics is quite close to normal. 


